When playing music, carrying on a telephone call, or listening to other audio content using a smartphone or other devices, another audio stream may “barge-in.” For example, a playback of stereo music may be interrupted by a response from a virtual assistant, or by other types of audio notifications or alerts received from a server or generated by the smartphone. It is desirable for the smartphone to provide a more pleasing listening experience to a user when there are multiple audio streams.